Integration of depth map device for adaptive lighting control

ABSTRACT

A light system includes at least one time-of-flight image sensor configured to generate at least one zone distance measurement. At least one control unit is configured to receive the at least one zone distance measurement and to generate at least one control signal based on the at least one zone distance measurement. At least one light unit is configured to adapt an output of the light unit based on the at least one control signal.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims priority to European Patent Application No.17157491.6, filed on Feb. 22, 2017, which application is herebyincorporated herein by reference.

TECHNICAL FIELD

Some embodiments relate to an apparatus, and in particular but notexclusively, to an apparatus integrating depth map sensor functionalitywith adaptive lighting control.

BACKGROUND

Lighting systems are known and widely implemented for many variousapplications. For example lighting systems may be used in industrial,commercial, residential, or other leisure environments. Such lightingsystems are rapidly being converted into ‘smart’ systems whereby acentral controller may be configured to receive inputs from sensors suchas passive infra-red sensors configured to determine a ‘warm’ bodyentering or moving within a room and control a lighting system to lightup a room. Such systems are currently limited in that they generally mayonly determine the presence or absence of ‘objects’ of interest withinthe field of view of the detector.

More complex systems may employ a camera or array of cameras configuredto capture images which may then be analyzed to determine the presenceof ‘objects’ and furthermore in some systems the type of the object, thedirection of motion of the object or similar. However the implementationof image sensors sensitive to capture images with sufficient detail toenable such processing and the implementation of suitable processingmeans to process the images significantly raises the cost or ‘bill ofmaterials’ of such systems and also increases the power consumption ofas such and the lighting system.

Furthermore lighting systems are further using more intelligent lightunits. For example ‘pixelated’ light unit can be configured to adapt oradjust the pattern (or throw) of the generated light as well as beingable to adapt or adjust the level of lighting. It is able to performthese operations by adjusting the light generated by pixels or byswitching on and off pixels within the lighting system. An example ofsuch a pixelated light unit is the OSRAM UX:3 technology.

These adaptive light systems have also been employed in automotive orvehicular light systems as well as conventional static systems. In suchsystems the ability to monitor the scene and allow the lighting to beadapted based on the results of the analysis is a goal for many researchprojects.

SUMMARY

According to a first aspect, a light system comprises at least onetime-of-flight image sensor configured to generate at least one zonedistance measurement. At least one control unit is configured to receivethe at least one zone distance measurement and to generate at least onecontrol signal based on the at least one zone distance measurement. Atleast one light unit is configured to adapt an output of the light unitbased on the at least one control signal.

The at least one time-of-flight image sensor may be physicallyintegrated within the at least one light unit.

The at least one time-of-flight image sensor may be physicallyintegrated within the at least one control unit.

The at least one time-of-flight image sensor may be further configuredto generate multiple zone distance measurements, wherein the zonedistance measurements may be arranged in a determined spatialarrangement.

The determined spatial arrangement may be at least one of a linear arrayof spatial measurement zones, a regular two dimensional array of spatialmeasurement zones, or an irregular two dimensional array of spatialmeasurement zones.

At least one light unit configured to adapt the output of the light unitbased on the at least one control signal may be configured to adapt atleast one of: a light intensity output based on the at least one controlsignal, a light color output based on the at least one control signal,and a light pattern output based on the at least one control signal.

The at least one light unit may be a pixelated light unit.

The at least one time-of-flight image sensor may comprise at least onearray of single photon avalanche diodes.

According to a second aspect, a method for controlling a light systemcomprises generating at least one zone distance measurement using atleast one time-of-flight image sensor, generating at least one controlsignal using at least one control unit having based on a received atleast one zone distance measurement, and adapting an output of at leastone light unit based on the at least one control signal.

The method may further comprise physically integrating the at least onetime-of-flight image sensor within at least one of: the at least onelight unit and the at least one control unit.

Generating at least one zone distance measurement may comprisegenerating multiple zone distance measurements, wherein the zonedistance measurements may be arranged in a determined spatialarrangement.

The determined spatial arrangement may be at least one of: a lineararray of spatial measurement zones; a regular two dimensional array ofspatial measurement zones; an irregular two dimensional array of spatialmeasurement zones.

Adapting an output of at least one light unit based on the at least onecontrol signal may comprise at least one of: adapting a light intensityoutput of the at least one light unit based on the at least one controlsignal, adapting a light color output of the at least one light unitbased on the at least one control signal, and adapting a light patternoutput of the at least one light unit based on the at least one controlsignal.

The at least one light unit may be a pixelated light unit.

The at least one time-of-flight image sensor may comprise at least onearray of single photon avalanche diodes.

BRIEF DESCRIPTION OF THE DRAWINGS

Some embodiments will now be described by way of example only and withreference to the accompanying Figures in which:

FIG. 1 illustrates a principle of the “Time of Flight” method fordetermining the distance to an object;

FIG. 2 shows an integrated time-of-flight sensor/light unit apparatusaccording to some embodiments;

FIG. 3 shows an example plan view of a sensor region arrangementgenerated by an example time-of-flight sensor within the integratedtime-of-flight sensor/light unit apparatus as shown in FIG. 2;

FIG. 4 shows a schematic view of a system implementing an integration oftime-of-flight sensors, light unit apparatus and a central control unitaccording to some embodiments;

FIG. 5 shows an example elevation view of a sensor region arrangementgenerated by an example integrated system shown in FIG. 4; and

FIG. 6 shows a schematic view of an example integrated system within anautomotive implementation.

DETAILED DESCRIPTION OF ILLUSTRATIVE EMBODIMENTS

The making and using of the present embodiments are discussed in detailbelow. It should be appreciated, however, that the present disclosureprovides many applicable inventive concepts that can be embodied in awide variety of specific contexts. The specific embodiments discussedare merely illustrative of specific ways to make and use the disclosedsubject matter, and do not limit the scope of the different embodiments.

The concept as employed herein is a system which integrates a suitablecontrollable light unit with a time-of-flight (ToF) image sensor. Insome embodiments the concept covers the implementation of atime-of-flight single photon avalanche diode (ToF SPAD) sensor and amultiple zone or region ToF SPAD sensor configured to determine not onlythe presence of objects within the lighting scene but also the distancefrom the lighting system to the object. This information can then beused by a control entity to adaptively control a light unit within thelighting system.

In other words the concept involves the analysis of a lighting scene bya (multiple zone) Time-of-flight image sensor and using the multi-zonedepth map information (on persons or objects within the scene) toprovide a more intelligent light adaptation. For example in someembodiments the multi-zone depth map information from a ToF SPAD sensormay be received by a controller to make intelligent decisions on how theroom or scene should be illuminated using a pixelated light unit.

Devices for determining the distance to objects using a method called“Time of Flight” (ToF) which is described herein is known. This methodcomprises sending a light signal towards the object and measuring thetime taken by the signal to travel to the object and back. Thecalculation of the time taken by the signal for this travel may beobtained by measuring the phase shift between the signal coming out ofthe light unit and the signal reflected from the object and detected bya light sensor. Knowing this phase shift and the speed of light enablesthe determination of the distance to the object.

Single photon avalanche diodes (SPAD) may be used as a detector ofreflected light. In general an array of SPADs is provided as a sensor inorder to detect a reflected light pulse. A photon may generate a carrierin the SPAD through the photo electric effect. The photo generatedcarrier may trigger an avalanche current in one or more of the SPADs inan SPAD array. The avalanche current may signal an event, namely that aphoton of light has been detected.

FIG. 1 illustrates the general principle of a “Time of Flight” method.In FIG. 1, a generator 10 (PULSE) provides a periodic electric signal(for example, square-shaped). This signal powers a light unit 12. Anexample of a light unit 12 may be a light-emitting diode, or any knownlighting device, for example, a laser diode. The signal coming out oflight unit 12 is transmitted towards an object 16 and is reflected bythis object. The reflected light signal is detected by a light sensor(CAPT) 18. The signal on sensor 18 is thus phase-shifted from the signalprovided by the generator by a time period proportional to twice thedistance to object 16. Calculation block 20 (DIFF) receives the signalsgenerated by generator 10 and by sensor 18 and calculates the phaseshift between these signals to obtain the distance to object 16. SPADshave the advantage of picosecond time resolution so are ideal for thedetector of a compact time of flight pixel.

Single-photon avalanche diodes or “SPADs,” also called Geiger modeavalanche photodiodes, have a reverse biased p-n junction in which aphoto-generated carrier can trigger an avalanche current due to animpact ionization mechanism. SPADs may be designed to operate with areverse bias voltage well above the breakdown voltage.

SPADs may be operated as follows. At an initial time, the diode isbiased to a voltage larger greater than its breakdown voltage. Thereception of a photon in the diode junction area starts a currentavalanche in the diode, which creates an electric voltage pulse on theanode. The diode is then biased back to a voltage smaller greater thanthe breakdown voltage, so that the SPAD reacts again to the reception ofa photon. SPADs can currently be used in cycles having reactivationperiods shorter than 10 ns. Thereby, SPADs can be used at high frequencyto detect objects at relatively short distances from the measurementdevice, for example, distances ranging from a few centimeters to a fewtens of centimeters. In different embodiments, different ranges may besupported.

Although most of the following examples are described with respect tothe integration of ToF image sensors within a residential, commercial orindustrial lighting system the integration of ToF image sensors to otherlighting systems may be implemented in other embodiments. For examplethe integration of ToF image sensors within an automotive lightingsystem for enabling effective control of adaptive automotive headlightsor in a cabin lighting environment can be implemented. Thus for examplein some embodiments the ToF image sensors can be used to determine anobject distance from the vehicle and the system adaptively control thelighting based on this information.

The integration of the ToF image sensor within the system in someembodiments may feature physically integrating the ToF image sensorwithin the light unit or within the control unit. In some embodimentsthe ToF image sensor may be physically separate from the light unit. Insome embodiments the integration of the ToF image sensor may be definedby a communication between the ToF image sensor and control unit and/orlight unit. The communication may be wired or wireless communication. Insome embodiments of the ToF image sensor can be configured tocommunicate with other ToF image sensors and/or control unit(s) and/orlight unit(s) via a communication network. A wireless communicationnetwork may communicate using radio frequency communication or lightfrequency communication.

FIG. 2 shows an example light system where the ToF image unit isphysically integrated within a light unit is shown in further detail. Inthis example a light unit (or light source) 201, which may in someembodiments be a pixelated light unit, is shown. Physically integratedwithin the light unit 201 is shown a multi-zone (or multi-region) ToFimage sensor 203. The multi-zone ToF image sensor 203 may be a SPADdevice which is configured with multiple arrays of SPADs or a singlearray of SPADs with multiple regions. Each of the arrays or regions maybe configured to determine ToF based distance estimates for separatespatial regions or zones. This is shown in FIG. 2 by the ToF imagesensor 203 having multi-zone field-of-views shown by cones 211, 213,215, 217, and 219.

Furthermore, in some embodiments, the light unit 201 further comprises acontrol unit which is configured to control the light unit (for example,to control the light output intensity, light pattern, light temperatureetc.) based on the information from the ToF image sensor.

In some embodiments the light unit 201 comprises multiple lightgenerating units which may map onto the ToF image sensor multi-zonefield of views. This map may for example be a 1:1 map, such that withrespect to FIG. 2 the light unit 201 comprises five separate lightgenerating units each with a projection pattern which matches anassociated ToF image sensor zone. However in some embodiments the lightunit output and the time-of-flight image sensor zones are not mapped oneach other using a one to one relationship.

In some embodiments the light unit 201 control unit comprises at leastone processor configured to run software code or is firmware which isconfigured to receive at least one input from the ToF image sensor 203.The processor may be configured to perform target or object presencedetection and/or target or object motion predication. The control unitmay then generate suitable control signals to enable the light unit toadapt its output. For example the light unit may switch on or off thelight generating units based on the ToF image sensor output such that anobject of sufficient height triggers the light generating unit to switchon (and thus reduce the number of false positive triggers).

Similarly in some embodiments the intensity level of the lightgenerating unit may be controlled such that the light generating unitgenerates a first light intensity when an object is detected close tothe light unit and a second light intensity when the object is detectedfar from the light unit. Thus in such an example a person standing closeto the light unit is not dazzled by a very strong light and a personstanding far from the light unit is provided with a sufficient amount oflight.

Furthermore in some embodiments the color or temperature of the lightgenerating unit may be controlled such that the light generating unitgenerates a first color when an object is detected at a first distanceand a different color when an object is detected at a second distance.Thus in such an example a ‘correct’ distance or position may bedetermined by a user of the system when the light changes to a defined‘correct’ color.

Where the light unit is a pixelated light unit or may be adapted togenerate controllable patterns of light in some embodiments thegenerated light pattern may be controlled such that the light generatingunit generates a light pattern based on the ToF image sensorinformation. For example in some embodiments the light pattern generatedmay illuminate the area around the object without directly illuminatingthe detected object and thus prevent the light system from dazzling theuser. Furthermore in some examples areas of a room with no target orobject can remain unlit by controlling the light pattern generated bythe light generating unit.

In some embodiments the controller can furthermore predict motion ofobjects from the ToF image sensor data and control the light generatingunits or light unit pattern such that the room or area is illuminated inregions where the object is about to move to and thus illuminate thepredicted path in front of the object. This for example with respect tothe example shown in FIG. 2, the light unit 201, having determined aperson entering passing through the far left zone 211 and entering thenear left zone 213 may be configured to illuminate the mid zone 215 inadvance of the person entering it. Furthermore as the person enters themid zone 215 the light unit may illuminate the near right zone 217 inadvance and switch off or reduce the illumination for the far left zone211.

FIG. 3 shows a plan view of an example of a multi-zone (or multi-region)array coverage. This arrangement includes 15 zones arranged in a 5columns by 3 rows regular array of zones. Thus in FIG. 3 there is showna first (bottom) row of zones 301, 303, 305, 307, and 309, a second(middle) row of zones 311, 313, 315, 317 and 319, and a third (upper)row of zones 321, 323, 325, 327, and 329. Thus in some embodiments theToF image sensor may be configured to determine ranges to objectsmeasured in a single dimension such as shown in FIG. 2 but also in twodimensions such as shown in FIG. 3.

Although the examples shown in FIGS. 2 and 3 show a ‘regular’arrangement of zones and that each zone has a similar ‘field-of-view’range it is understood that the ToF image sensor can be configured todetect the range or distance from the sensor for zones which arearranged in an irregular arrangement. Furthermore although FIG. 3 showsthat each zone is substantially equal in size in some embodiments thezones may be larger or smaller than other zones covered by the ToF imagesensor and may have differing shaped ‘field-of-view’ configurations.

FIG. 4 shows a schematic view of an integrated light system comprising adistributed network of ToF image sensors, control unit and light units.As shown in FIG. 4 an array of ToF image sensors 401, 403, 405, and 407are shown coupled to a control unit 411. The ToF image sensors may bethe same or be different. Furthermore each ToF image sensor may have oneor multi-zone detection capability. The information from each of the ToFimage sensors 401, 403, 405 and 407 can be passed to the control unit411. The coupling between the ToF sensor units and the control unit maybe any suitable data connection and as discussed earlier be wired,wireless, radio frequency or light frequency based.

The controller unit 411 may be physically separate from the ToF imagesensors or be physically integrated within one or more of the ToF sensorunits and/or the light units. In some embodiments the control unit 411functionality may be provided by a ‘cloud’ based server. As describedpreviously the controller unit 411 may be configured to receive the ToFimage sensor information and determine lighting parameters such aspattern, intensity, color or otherwise and generate control signals withmay be passed to the light units 421, 423, and 425 shown in FIG. 4 via aseparate communications network. In some embodiments, the communicationsnetwork coupling the light units with the control unit is the same asthe communications network coupling the control unit with the ToF imagesensors.

The light units 421, 423 and 425 may be light units configured toreceive suitable lighting control signals and then adapt the light orillumination generated by the light units (or light generating units)based on the control signals. The light units in some embodiments may belocated within the same room or scene or in different rooms or scenes.Thus for example in some embodiments the light unit in a room may becontrolled based on information from a ToF image sensor located andmonitoring a neighboring room such that the light in the room may beswitched on just before a person enters the room.

FIG. 5 shows a further example of an integrated lighting system. Theintegrated lighting system shows a control unit 501 which is networkedto an array of light units and Time-of-Flight image sensors. The lightunits 511, 513, 515, and 517 each have a light unit projection range(shown in FIG. 5 by like light unit 511 having the cone 531). The lightunits are coupled via a communications network to the control unit 501.The ToF image sensors 521, 523, 525 furthermore are configured tocommunicate with the control unit 501 over the communications network.The ToF image sensors are arranged to be physically discrete from thelight units and located between the light units such that a first ToFimage sensor 521 is located between the light units 511 and 513, asecond ToF image sensor 523 is located between the light units 513 and515 and a third ToF image sensor 525 is located between the light unit515 and 517. This arrangement generates a ToF image sensor field of viewwhich covers the majority of the light unit illuminated areas.

FIG. 6 shows an example of an integrated light system suitable for avehicular or automotive application. A vehicle 600 may be equipped witha light unit 601 which is integrated with a Time-of-Flight image sensor603 such that the ToF image sensor 603 is configured to monitor the areain front of the vehicle 600 and determine at least one distancemeasurement from the vehicle 600. The ToF image sensor 603 may beconfigured to generate multi-zone distance determination.

The ToF image sensor 603 information is passed to a controlfunctionality (which may be separate from or physically integratedwithin the ToF image sensor 603 and/or the light unit) and which isconfigured to adjust or adapt the output from the light unit 601. Forexample in some embodiments the light unit 601 may be configured toadjust or adapt the illumination pattern 605 (or light cone generated bythe light unit) with respect to shape, intensity, or color based on theinformation from the ToF image sensor 603.

Some embodiments may use other sensors, instead of SPADs. These sensorsmay be integrating photo-sensitive elements capable of receiving lightintensity, time of arrival, frequency or phase or amplitude/intensitymodulation, wavelength (color) or other information.

It should be appreciated that the above described arrangements may beimplemented at least partially by an integrated circuit, a chip set, oneor more dies packaged together or in different packages, discretecircuitry or any combination of these options.

Various embodiments with different variations have been described hereabove. It should be noted that those skilled in the art may combinevarious elements of these various embodiments and variations.

Such alterations, modifications, and improvements are intended to bepart of this disclosure, and are intended to be within the scope of thepresent invention. Accordingly, the foregoing description is by way ofexample only and is not intended to be limiting. The present inventionis limited only as defined in the following claims and the equivalentsthereto.

What is claimed is:
 1. A light system comprising: a time-of-flight imagesensor configured to generate a zone distance measurement; a controlunit configured to receive the zone distance measurement and to generatea control signal based on the zone distance measurement; and a lightunit configured to adapt an output of the light unit based on thecontrol signal, wherein the time-of-flight image sensor comprises adistributed network of time-of-flight image sensors and the light unitcomprises a distributed network of light units, wherein each of thedistributed network of time-of-flight image sensors is arranged to bephysically discrete from each of the distributed network of light unitsand located between adjacent ones of the distributed network of lightunits.
 2. The light system as claimed in claim 1, wherein thetime-of-flight image sensor is physically integrated within the lightunit.
 3. The light system as claimed in claim 1, wherein thetime-of-flight image sensor is physically integrated within the controlunit.
 4. The light system as claimed in claim 1, wherein thetime-of-flight image sensor is further configured to generate multiplezone distance measurements, wherein the zone distance measurements arearranged in a determined spatial arrangement.
 5. The light system asclaimed in claim 4, wherein the determined spatial arrangement comprisesa linear array of spatial measurement zones.
 6. The light system asclaimed in claim 4, wherein the determined spatial arrangement is aregular two dimensional array of spatial measurement zones.
 7. The lightsystem as claimed in claim 4, wherein the determined spatial arrangementis an irregular two dimensional array of spatial measurement zones. 8.The light system as claimed in claim 1, wherein the light unit isconfigured to adapt a light intensity output based on the controlsignal, a light color output based on the control signal, or a lightpattern output based on the control signal.
 9. The light system asclaimed in claim 1, wherein the light unit is a pixelated light unit.10. The light system as claimed in claim 1, wherein the time-of-flightimage sensor comprises an array of single photon avalanche diodes. 11.The light system as claimed in claim 1, wherein each of the distributednetwork of time-of-flight image sensors is configured to produce a zonedistance measurement corresponding to a first field of view, whereineach of the distributed network of light units is configured toilluminate a second field of view, wherein the first field of view ofeach of the distributed network of time-of-flight image sensors overlapswith two of the second field of views of adjacent light units of thedistributed network of light units.
 12. A light system comprising: amulti-zone time-of-flight image sensor configured to generate aplurality of zone distance measurements, each zone distance measurementcorresponding to a field of view; and a light unit with a plurality oflight sources, each light source corresponding to field of view andconfigured to output light that includes characteristics based on thezone distance measurement of the corresponding field of view, whereinthe light unit comprises a pixelated light unit and wherein themulti-zone time-of-flight image sensor is embedded inside the pixelatedlight unit.
 13. The light system as claimed in claim 12, furthercomprising a control unit operationally coupled to the multi-zonetime-of-flight image sensor and the light unit.
 14. The light system asclaimed in claim 13, wherein the control unit is physically integratedwith the multi-zone time-of-flight image sensor.
 15. The light system asclaimed in claim 13, wherein the distributed network of light units arecoupled through a communication network to the control unit.
 16. Thelight system as claimed in claim 12, wherein the light system is part ofa headlight for an automotive vehicle.
 17. The light system as claimedin claim 12, wherein the multi-zone time-of-flight image sensor isfurther configured to generate multiple zone distance measurements,wherein the zone distance measurements are arranged in an non-linearspatial arrangement.
 18. The light system as claimed in claim 12,wherein the multi-zone time-of-flight image sensor is further configuredto generate multiple zone distance measurements, wherein the zonedistance measurements are arranged in a two-dimensional array of spatialarrangement.
 19. A method for controlling a light system, the methodcomprising: generating a zone distance measurement using atime-of-flight image sensor; generating a control signal based on thezone distance measurement; and adapting an output of a light unit basedon the control signal, wherein the time-of-flight image sensor comprisesa distributed network of time-of-flight image sensors and the light unitcomprises a distributed network of light units, wherein each of thedistributed network of time-of-flight image sensors is arranged to bephysically discrete from each of the distributed network of light unitsand located between adjacent ones of the distributed network of lightunits.
 20. The method as claimed in claim 19, wherein generating thezone distance measurement comprises generating multiple zone distancemeasurements, wherein the zone distance measurements are arranged in adetermined spatial arrangement.
 21. The method as claimed in claim 20,wherein the determined spatial arrangement is a linear array of spatialmeasurement zones, a regular two dimensional array of spatialmeasurement zones, or an a regular two dimensional array of spatialmeasurement zones.
 22. The method as claimed in claim 19, whereinadapting the output of the light unit comprises adapting a lightintensity output of the light unit.
 23. The method as claimed in claim19, wherein adapting the output of the light unit comprises adapting alight color output of the light unit.
 24. The method as claimed in claim19, wherein adapting the output of the light unit comprises adapting alight pattern output of the light unit.